BEGINRELEASENOTES

• Add improvements to k4run: clean up code, use the logging module, parse all the arguments using argparse, sort display of options

ENDRELEASENOTES

• Parsing is done using argparse only instead of the previous combination of manual parsing and argparse. I think this is cleaner and if the arguments are wrong it will show the usage automatically (before there was a explicit testing of this)
• Code was cleaned up a bit, for example there were very deeply nested if/for that can be changed a bit. Indentation is (and was) inconsistent so if we agree on 2 or 4 spaces I can change that; it makes the diff very hard to read so we can keep it for the last change.
• Previously the display of options wouldn't be sorted and would be different each time because there was a list(set( that has undetermined ordering, now it will always be sorted in the same order as they were defined in the file (for python >= 3.7, since it is a one liner in that case).
• Now using logging for messages as per Use python logger instead of print() in k4Run #50. Right now there is a [k4run] for the messages that come from k4run but this is a placeholder. For example for the Gaudi processes the format looks like [ MESSAGE "VXDEndcapDigitiser"] and many things can be done like adding the date, the level of the message, etc. I don't have a strong opinion. This is how it looks now:

[k4run] LcioEvent --> MyAIDAProcessor --> EventNumber --> InitDD4hep --> Config --> OverlayFalse --> VXDBarrelDigitiser --> VXDEndcapDigitiser --> InnerPlanarDigiProcessor --> InnerEndcapPlanarDigiProcessor --> OuterPlanarDigiProcessor --> OuterEndcapPlanarDigiProcessor --> MyConformalTracking --> ClonesAndSplitTracksFinder --> Refit --> MyDDCaloDigi --> MyDDSimpleMuonDigi --> MyDDMarlinPandora --> LumiCalReco --> MergeRP --> MergeClusters --> MyClicEfficiencyCalculator --> MyRecoMCTruthLinker --> MyTrackChecker --> CLICPfoSelectorDefault_HE --> CLICPfoSelectorLoose_HE --> CLICPfoSelectorTight_HE --> CLICPfoSelectorDefault_LE --> CLICPfoSelectorLoose_LE --> CLICPfoSelectorTight_LE --> RenameCollection --> VertexFinder --> JetClusteringAndRefiner --> Output_REC --> Output_DST
[k4run] Option name: EventDataSvc.OutputLevel EventDataSvc.OutputLevel 0
[k4run] Option name: LcioEvent.OutputLevel LcioEvent.OutputLevel 4
[k4run] Option name: LcioEvent.Files LcioEvent.Files ['ttbar.slcio']
[k4run] Option name: InitDD4hep.OutputLevel InitDD4hep.OutputLevel 4
[k4run] Option name: InitDD4hep.ProcessorType InitDD4hep.ProcessorType InitializeDD4hep

I ran clicReconstruction.py on the same file and got similar results

Other related issues:

• k4DataSvc seg faults if input file does not exist #89 I could reproduce with the podio reader but this has been fixed in Deal with missing files in the readers AIDASoft/podio#355. With the LCIO reader it exits without crashing. Can we close it?
• Do not attach gdb by default #90 I checked and certainly by default the option to attach gdb is set to false for Gaudi. I could reproduce this issue with the podio reader before Deal with missing files in the readers AIDASoft/podio#355 (by trying to open a non-existent file, now fixed), where TFile.Open fails and pulls gdb. It was very very slow. Interestingly I ran with everything local (without cvmfs) and it took just a couple of seconds to display the backtrace, so cvmfs is at least partly responsible for the slowness. Already closed.
• Remove --ncpus flag in k4run until it is actually supported #72 Using --npcus with a number different from 0 fails with:

  File "/cvmfs/sw.hsf.org/spackages6/gaudi/36.5/x86_64-centos7-gcc11.2.0-opt/z2prd/python/Gaudi/Main.py", line 476, in run
    result = self.runParallel(ncpus)
  File "/cvmfs/sw.hsf.org/spackages6/gaudi/36.5/x86_64-centos7-gcc11.2.0-opt/z2prd/python/Gaudi/Main.py", line 561, in runParallel
    Parall = gpp.Coord(ncpus, c, self.log)
  File "/cvmfs/sw.hsf.org/spackages6/gaudi/36.5/x86_64-centos7-gcc11.2.0-opt/z2prd/python/GaudiMP/GMPBase.py", line 1378, in __init__
    sub = Subworker(
  File "/cvmfs/sw.hsf.org/spackages6/gaudi/36.5/x86_64-centos7-gcc11.2.0-opt/z2prd/python/GaudiMP/GMPBase.py", line 834, in __init__
    GMPComponent.__init__(
  File "/cvmfs/sw.hsf.org/spackages6/gaudi/36.5/x86_64-centos7-gcc11.2.0-opt/z2prd/python/GaudiMP/GMPBase.py", line 473, in __init__
    self.msgFormat = self.config["MessageSvc"].Format
KeyError: 'MessageSvc'